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ABSTRACT 


Human Systems Integration (HSI) is a process designed to reduce life- 
cycle costs and improve system performance by considering human-related 
domains. Acquisition specialists lack objective, quantitative research findings on 
which to base trade-off analyses. This thesis used eight fiscal years of historical 
safety (mishaps), manpower (manning levels) and system performance (SORTS) 
data on the U.S. Navy’s Frigates, collected from computer databases of routine 
reports, to explore relationships in an existing notional model of HSI. Three 
hypotheses were tested: there is a negative relationship between manning and 
SORTS levels; there is a negative relationship between manning and mishaps; 
and there is a positive relationship between mishaps and SORTS levels. No 
significant relationships were found between SORTS levels and manning or 
mishaps. When all of the ships were ranked for each month based on percent of 
total manning and number of reportable mishaps, a positive correlation 
(Spearman’s rho = 0.4194, p-value = 0.0294) was found corresponding to a 
negative relationship between manning levels and mishap rates. More detailed 
research is needed to isolate the relationship between manning levels and 
mishap rates from numerous other influences and any noise that may be present 


in the data set. 
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EXECUTIVE SUMMARY 


Human Systems Integration (HSI) is a process designed to reduce life- 
cycle costs and improve system performance by considering human-related 
domains. All Department of Defense (DoD) acquisition programs are required to 
consider HSI. The Navy’s HSI guidance includes modernization programs of 
legacy systems. However, acquisition specialists lack objective, quantitative 
research findings on which to base trade-off analyses for developmental or 


legacy systems. 


This thesis used eight fiscal years of historical safety (mishaps), 
manpower (manning levels) and system performance (SORTS) data on the U.S. 
Navy's Oliver Hazard Perry (FFG-7) Class Frigates, collected from computer 
databases of routine reports, to explore relationships in an existing notional 
model of HSI. The three hypotheses tested were, there is a negative relationship 
between manning and SORTS levels, there is a negative relationship between 
manning and mishaps, and there is a positive relationship between mishaps and 
SORTS levels. 


No significant relationships were found between SORTS levels and 
manning or mishaps. When all of the ships were ranked from 1 (best), to 27 
(worst), for each month of the study period based on percent of total manning 
and number of reportable mishaps, a positive correlation (Spearman’s rho = 
0.4194, p-value = 0.0294) was found corresponding to a negative relationship 


between manning levels and mishap rates. 


No practically significant differences were found in manning, personnel, 
performance or safety based on date, ships, homeport, or region. This means 
that data collected on a sample of ships should be applicable to all Navy 
Frigates. However, there were significant differences found in manning levels 


between the Navy Reserve ships and active component ships. 


XV 


The data analyzed for this thesis were insufficient to validate all the 
relationships in NPS’s notional HSI trade space tool and more detailed research 
is needed to isolate the relationship between manning levels and mishap rates 
from numerous other influences and any noise that may be present in the data 


set. 


HSI, though required by law, is often confused as just HFE or just SE. 
There remains a need for shared representations of the HSI trade space to 
facilitate Communication between what are often isolated domains. While the 
data collected for this thesis were insufficient to validate all the relationships in 
one notional trade space tool, finding a negative relationship between manning 
and mishap rates is a substantial step forward. Just as this thesis advanced 
previous work, further research is still needed to fully develop NPS’s HSI TST to 


maturity. 
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I. INTRODUCTION 


The stated goal of Human Systems Integration (HSI) is to reduce life-cycle 
costs and improve system performance by considering human-related domains. 
All Department of Defense (DoD) acquisition programs are required to consider 
HSI. However, there is a dearth of evidence to define relationships among 
domains. This thesis is the first empirical exploration of the relationships between 


HSI domains and system performance, based on historical data. 
A. HISTORY OF HUMAN SYSTEMS INTEGRATION (HSI) 


The DoD defines HSI as a process designed to “optimize total system 
performance, minimize total ownership costs, and ensure that the system is built 
to accommodate the characteristics of the user population that will operate, 
maintain, and support the system” (Department of Defense Instruction [DoDI] 
5000.2, 2004, p. E7-1). HSI encompasses seven distinct domains: human factors 
engineering (HFE), personnel, habitability, manpower, training, environmental 
safety and occupational health (ESOH), and survivability (DoDI 5000.2, 2004). 


At the most basic level, human considerations are as old as tool making; 
older than civilization itself. Human-centered design considerations have been 
around since the first humanoid chose which rock to shape into a hand ax. Over 
the centuries, systems became more complex and mass production replaced 
custom production, increasing the challenge of designing systems to 


accommodate human users. 


All of the individual domains of HSI predate the formation of HSI as a 
discipline. Academic books on the subject describe HFE as the initial domain of 
HSI. The addition of the other domains, and the consideration of the interactions 
between the domains, produces the HSI process. Research into pilot error in 
World War II was instrumental in formalizing work in HFE into a discipline (Pew & 
Mavor, 2007; Simpson, 2003). In the foreword to the Handbook of Human 


1 


Systems Integration, Booher describes HSI as the third milestone of 
human-focused considerations within the U.S. Defense community. He lists the 
first milestone as the development of HFE and the second milestone as 
implementation of the U.S. Army’s Manpower and Personnel Integration 
(MANPRINT) program (Booher, 2003). 


B. RATIONALE FOR THE STUDY 


i; The Human Systems Integration (HSI) Mandate 


In 2003, the United States Congress, through Department of Defense 
Directive (DODD) 5000.1, mandated the inclusion of HSI in all DoD acquisition 
programs. The primary sources of HSI requirements are The Defense Acquisition 
System, DoDD 5000.1, and Operations of the Defense Acquisition System, 
DoDI 5000.2. Chapter Six of the Defense Acquisition Guidebook (DAG) provides 
nonbinding guidance on implementing the mandates in DoDD 5000.1 and 
DoDI 5000.2. 


The most cost effective way to implement the congressional mandate is to 
consider HSI trade-offs from the beginning of the acquisition process. However, 
acquisition specialists lack objective, quantitative research findings on which to 


base trade-off analyses for developmental or legacy systems. 
2. Human Systems Integration (HSI) in Legacy Systems 


DoDD 5000.1 applies to all New DoD acquisition programs (DoDD 5000.1, 
2007). Secretary of the Navy (SECNAV) Instruction 5000.2C and Naval Sea 
Systems Command (NAVSEA) Instruction 3900.8A expand the applicability of 
HSI to include modernization, disposal, and demilitarization of new and existing 
programs. Both of these Navy instructions recognize the need for HSI to evaluate 
alterations to legacy systems. Navy ships are not static objects; they are large 
and complex systems of systems, operating in a dynamic global environment. 


Predictions are that in the next ten years, “HSI personnel will become the experts 


2 


in issue tracking and resolution as systems are now never finished” (Pew & 
Mavor, 2007, p. 286). Currently, program managers struggle to predict and plan 
for HSl-related considerations (total life-cycle costs, utilization, operational 
environment) in the initial acquisition process. This thesis seeks to further 
develop a model that can evaluate the impact of alterations to developmental and 
legacy programs. Alterations to legacy programs could include any or all of the 
doctrine, organization, training, materiel, leadership and education, personnel, 
and facilities (DOTMLPF) factors. 


The absence of attention to HSI throughout the full life cycle is likely due 
to the argument of a low return on investment. Changes and improvements in 
legacy systems are severely constrained by the existing HFE level. For example, 
HFE, through physical space and acceptable habitability levels, sets the upper 
limit of manning on a ship. Lower limits are set through the tasks (manpower) 
and skills (personnel) needed to operate the ship. This leaves HFE and training 
as the primary avenues to alter a system’s manpower, personnel, habitability, 
safety, and survivability. Compared to the initial concept development of a novel 
system, legacy systems have a restricted trade space and higher cost for 


marginal improvements. 


The low-return-on-investment argument overlooks two opportunities. 
Since DoD systems are large, complex, and operate in a dynamic environment, 
they change constantly. Using HSI will improve the quality of modernization 
efforts. Second, identifying and capturing lessons learned, even those not cost 
effective to fix in the current system, will improve future systems under 
development. In addition to evaluating initial design decisions of legacy systems, 
program managers would benefit from careful studies of how systems evolve 


and operate. 


C. OBJECTIVE 


1. Scope 


This thesis seeks to explore relationships among HSI domains. A subset 
of the relationships from the Naval Postgraduate School’s (NPS) Trade-Space 
Tool (TST) is studied, using a retrospective analysis of historical data. This thesis 
is built on a theoretical foundation laid by an NPS thesis, “Toward an Improved 
Method of HSI Evaluation in Defense Acquisition” (Simpson, 2006). The fleet of 
U.S. Navy Frigates served as a proof of concept to explore the relationships 


among HSI domains. 
2. Limitations 


This thesis focused on a single class of ship in an effort to improve 
accuracy at the expense of generalization to other platforms. The U.S. Navy’s 
Oliver Hazard Perry Class Frigate (FFG-7) is an ideal initial platform to analyze 
for several reasons. It is the smallest and simplest ship in the cruiser and 
destroyer (CRUDES) class. The decision to focus on the U.S. Navy Frigate 
simplified the amount of data to gather, encode, and analyze. Two of the four 
surface ship classes currently in development are in the CRUDES class—the 
Littoral Combat Ship and the next generation Destroyer. Minesweepers and 
patrol craft, though smaller and simpler systems, might not provide as much 


insight into these current systems under development. 


The data collected were limited to measures available on computer 
databases from routine reports. This study did not attempt to collect data from 
the ships directly, but instead relied on data already stored in a variety of 
repositories. The complexity of the problem space necessitated numerous 
simplifications. Many of the terms used in DoDD 5000.1 are vague and context 
related, making them challenging to measure. Chapter III, Section G of this thesis 
details how the variables of manpower, HFE, habitability, survivability, safety, 


and performance were quantified for the purposes of this thesis effort. 
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ll. LITERATURE REVIEW 


A. OVERVIEW 


This chapter provides a general overview of the HSI literature. Sections B 
and C discuss the chain of guidance, in the order of precedence, that applies to 
the United States Navy. This chain of guidance represents just one of many 
document chains branching off the DoD HSI guidance. Section D discusses and 
summarizes two books on HSI. Section E concludes the HSI literature review 
with a summary of how all the sections build a picture of the current state of HSI 
within the DoD. Section F highlights the relationships among the HSI domains, 
with a focus on safety. Section G introduces NPS’s concept of HSI. Section H 
describes the two of the three data sources used in this thesis: the official U.S. 
Navy mishap reports and Status of Resources and Training System (SORTS) 


reports. 
B. DEPARTMENT OF DEFENSE (DoD) HSI GUIDANCE 


DoDD 5000.1, The Defense Acquisition System, and DoDI 5000.2, 
Operation of the Defense Acquisition System, reference each other and together 
provide the superseding guidance for all DoD acquisition programs. The Defense 
Acquisition Guidebook (DAG) contains suggestions and recommended best 
practices for implementing DoDD 5000.1 and DoDI 5000.2 (DAU Website, 2004). 
The Chairman, Joint Chiefs of Staff Instruction (CJCSI) 3170.01F and the 
Chairman, Joint Chiefs of Staff Manual (CJCSM) 3170.01C provide guidance for 
Joint Capabilities Integration and Development System (JCIDS) acquisition 
programs (CJCSI 3170.01F, 2007, CJCSM 3170.01C, 2007). 


1. Department of Defense Directive (DODD) 5000.1 


DoDD 5000.1, Section 1.2 states, “This directive, along with reference (b) 
[DoDI 5000.2], provides management principles and mandatory policies and 
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procedures for managing all acquisition programs” (DODD 5000.1, 2003, p. 2). 
Section 2, Applicability and Scope, states that the DoDD 5000.1 applies to all 


acquisition programs within the Defense Department. 


The only direct reference to HSI is Enclosure 1.29, Total Systems 
Approach, which directs program managers to “apply human systems integration 
to optimize total system performance (hardware, software, and human), 
operational effectiveness, and suitability, survivability, safety, and affordability” 
(DoDD 5000.1, 2003, p. E1-29). Enclosure 1.29 goes on to stress total ownership 
and life-cycle considerations. When viewed online, the words “human system 


integration” hyperlink to Chapter 6 of the DAG. 
2. Department of Defense Instruction (DoDI) 5000.2 


The DoDI 5000.2 purpose section states, “This instruction: implements 
reference (c) [DoDD 5000.1], the guidelines of references (d) and (e), and current 
laws” (DoDI 5000.2, 2003, p. 2). DoDI 5000.2 has the same applicability as 
DoDD 5000.1, to all DoD acquisition programs, with the additional guidance that 
some requirements only apply to Major Defense Acquisition Programs (MDAPs) 
and Major Automated Information System (MAIS) programs (DoDI 5000.2, 2003). 


Enclosure 7, Human Systems Integration (HSI), directs the program 


manager to 


. have a comprehensive plan for HSI in place early in the 
acquisition process to optimize total system performance, minimize 
total ownership costs, and ensure that the system is built to 
accommodate the characteristics of the user population that will 
operate, maintain, and support the system. (DoDI 5000.2, 2003, 
p. E7) 


The enclosure then lists seven HSI domains, with direction on how the 


program managers are to address each one. DoDI! 5000.2, Enclosure 7, is 


included as Appendix B of this thesis. Although it implies the need to make 
trade-offs, DoDI 5000.2 does not direct how to “optimize performance” to 


“minimize total ownership costs.” 


In addition to Enclosure 7, Section 3.7.1.1 mentions HSI as one of a 
dozen considerations during the system development and demonstration phase. 
Section 3.9.2.2, under Sustainment, does not mention HSI by name, but does 
direct program managers to consider the HSI domains of manpower, personnel, 
training, HFE, habitability, environmental, safety, and occupational health 
(ESOH), and survivability. 


3. Defense Acquisition Guidebook (DAG) 
The DAG is an online document 


designed to complement [the DoDD 5000.1 and_ the 
DoDI 5000.2] by providing the acquisition workforce’ with 
discretionary best practice that should be tailored to the needs of 
each program . . . . All chapters contain non-mandatory staff 
expectations for satisfying the mandatory requirements in 
DoD Instruction 5000.2. (DAG, 2004, {| foreword) 


Chapter 6 of the DAG is the primary source of high-level guidance on HSI. 
The overview states, “DoD acquisition policy [DoDD 5000.1] requires optimizing 
total system performance and minimizing the cost of ownership through a ‘total 
system approach’ to acquisition management” (DAG, 2004, {| 6.0). Section 6.0.1, 
Purpose, defines HSI as an element of the systems engineering process, whose 
understanding will allow Program Managers to effectively and affordably 
integrate human capabilities and limitations in system design and development 
(DAG, 2004). 


Chapter 6 has six subsections. Section 6.1, Total System Approach, 
briefly reviews DoDD 5000.1 directed total system approach. It highlights the 
need to consider maintenance, support, and training as parts of the total system. 


Section 6.2 has seven subsections, each one describing an HSI domain. 
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Section 6.3 provides planning details for the HFE domain. Section 6.4 provides 
advice on HSI implementation. Section 6.5 describes the human considerations 
associated with resource estimating and planning. The last section of Chapter 6, 
Section 6.6, provides two reference listings for additional information 
(DAG, 2004). 


Section 6.4 articulates a vision of what HSI is and, in general, how to 


conduct it. 


The key to a successful HSI strategy is integration. To optimize 
total system performance and determine the most effective, 
efficient, and affordable design entails trade studies both within the 
HSI elements and between the HSI elements and the system 
platform. (DAG, 2004, {| 6.4) 


It goes on to recommend that Program Managers integrate HSI 
parameters/requirements in the Initial Capabilities Document (ICD), Capability 
Development Document (CDD), and Capability Production Document (CPD). In 
addition to Chapter 6, recommendations to apply HSI appear throughout the 
DAG, from initial capabilities through final test and evaluation to life-cycle 


logistics. 


4. Chairman, Joint Chiefs of Staff Instruction (CJCSI) 
3170.01 Series 


The CJCSI 3170.01F, Joint Capabilities Integration and Development 
System, is a 59-page document that establishes the policies and procedures of 
the JCIDS (CJCSI 3170.01F, 2007). The instruction mentions HSI only five 
times—twice in Enclosure B, Policy, and three times in the Glossary. Enclosure 


B states: 


DOTMLPF [doctrine, organization, training, materiel, leadership and 
education, personnel, and facilities] includes analysis of the entire 
life cycle, including the sustainment; environment, safety, and 
occupational health (ESOH); and all human systems integration 
(HSI) domains. (CJICSI 3170.01F, 2007, p. B-1) 
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Later, Enclosure B mentions HSI as one consideration when identifying 
capability gaps and potential materiel and nonmaterial solutions (CJCSI 
3170.01F, 2007). The Glossary definition of HSI references DoDI 5000.2 and 
adds that it includes the integrated and comprehensive analysis, design, and 
assessment of requirements, concepts, and resources for system manpower, 
personnel, training, safety and occupational health, habitability, personnel 
survivability, and HFE. The Glossary definition of “nonmaterial solutions” refers to 
HSI under policy changes. The Glossary also lists HSI as a consideration for 
“supportability” (CICSI 3170.01F, 2007). 


5. Chairman, Joint Chiefs of Staff Manual (CJCSM) 3170.01 Series 


CJCSM 3170.01C, Operation of the Joint Capabilities Integration and 
Development System, is a 149-page document that 

.. . sets forth guidelines and procedures for operation of the Joint 

Capabilities Integration and Development System (JCIDS) 

regarding the development and staffing of JCIDS documents in 


support of reference (a) [CJCSI 3170.01 series]. (CJCSM 
3170.01.C, 2007, p. 1) 


Appendix A to Enclosure E, Capability Development Document Format, 
and Appendix A to Enclosure F, Capability Production Document Format, each 
mention HSI twice. Under “Other DOTMLPF and Policy Considerations,” it 
advises discussion of HSI considerations that have a major impact on system 
effectiveness, suitability, and affordability. Under “Other System Attributes,” it 
mentions HSI as one factor to “consider as _ appropriate’ (CJCSM 
3170.01C, 2007). 


6. Conclusions on Department of Defense (DoD) Guidance 


The DoD instructions are high-level and broad. While the DAG provides 
excellent advice, it is nonbinding. The CJCSI and CJCSM mention HSI as a 
consideration, but do not define it or provide direction about how to 


employ it. 


C. UNITED STATES NAVY HUMAN’ SYSTEMS __ INTEGRATION 
(HSI) GUIDANCE 


This section looks at the chain of instructions that pertain to the 
implementation of HSI for U.S. Navy systems. These instructions provide 
Clarification and further guidance to the DoD and CJCS guidance already 
discussed. This section presents the instructions by order of precedence, from 
highest to lowest. At each level, the documents provide requirements in addition 


to the requirements in the documents they reference. 


1. Secretary of the Navy (SECNAV) Instruction 5000.2 Series 


SECNAV Instruction 5000.2D, Implementation and Operation of the 
Defense Acquisition System and the Joint Capabilities Integration and 
Development System, is a 195-page instruction that includes guidance for HSI. It 
issues requirements for major and nonmajor defense acquisition and information 
technology programs for the Department of the Navy (DON) (SECNAV 5000.2D, 
2008). It also cites the DAG as a discretionary companion to the instruction. 


Section 1.7.k, under Responsibilities, identifies the Director, Total Force 
Requirements Division, (CNO [N12]), as the single governing authority for HSI 
policy, requirements, and resources. The N12 is directed to participate in the 
identification of enterprise Manpower, Personnel, Training (MPT) and education 
shortfalls, and to investigate innovative approaches and solutions to optimize 


manpower and improve performance (SECNAV 5000.2D, 2008). 


The primary HSI guidance in SECNAV 5000.2D is in Enclosure 7, 
Systems Engineering and Human Systems Integration. It directs program 
managers to include HSI in a systems engineering process used to translate 
operational requirements/capability needs into a system solution (SECNAV 
5000.2D, 2008). Section 7.1.6.2, explicitly directs information technology (IT) 
systems to address interoperability and HSI considerations (SECNAV 5000.2D, 
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2008). Section 7.2.1, HSI in Acquisition, specifically addresses HSI and defines 
the scope of acquisition programs, to include modifications to legacy systems 
(SECNAV 5000.2D, 2008). 


Sections 7.2.2 through 7.2.5 of SECNAV 5000.2D provide one paragraph 
each on how to consider the HSI domains. Section 7.2.2 lists MPT as a single 
factor. The other three sections are 7.2.3, HFE; 7.2.4, Personnel Survivability; 
and 7.2.5, Habitability. Section 7.3 addresses Environmental, Safety, and 
Occupational Health (ESOH) considerations and is seven pages long, including 
two full-page tables (SECNAV 5000.2D, 2008). 


HSI is briefly mentioned in Section 2.1.2.3 under the discussion of 


manpower requirements. 


DOTMLPF analyses, conducted as the first step in the Functional 
Solutions Analysis, shall address all eight Human Systems 
Integration (HSI) domains including Environmental, Safety, and 
Occupational Health (ESOH). (SECNAV 5000.2D, 2008, p. E2-10) 


Section 3.4 is Acquisition Strategy. Section 3.4.7.1, Human Systems 
Integration (HSI), mandates an acquisition strategy that summarizes HSI 
planning, including how the program will meet HSI program requirements and 
standards (SECNAV 5000.2D, 2008). 


2. Chief of Naval Operations Instruction 5310.23 Series 


OPNAVINST 5310.23 is currently in draft format. If signed, it will reference 
SECNAVINST 5000.2D and will replace reference (a), Chief of Naval Operations 
Guidance for 2004, of the Naval Sea Systems Command (NAVSEA) Instruction 
3900.8 Series. 


3. Naval Sea Systems Command (NAVSEA) _Instruction 
3900.8 Series 


NAVSEA Instruction 3900.8A is Human System Integration (HSI) Policy in 


Acquisition and Modernization Programs (NAVSEA 3900.8A, 2005). This 
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16-page document expands the definition of acquisition programs to include 


research and development, modernization, and disposal programs. 


Section 6 of NAVSEA 3900.8A, Responsibilities, outlines the roles and 
tasks for each level from the NAVSEA 03 to Ship Design Managers. Section 6 
directs NAVSEA Deputy Commanders to “ensure manpower and _ training 
trade-offs studies are conducted and documented . . . to minimize manning and 
training costs” (NAVSEA 3900.8A, 2005, p. 4). 


Enclosure 1 provides the first detailed guidance on how to develop and 
implement an HSI plan. Enclosure 1 explicitly states that an interdependent 
relationship exists between the HSI domains, and trade-offs made in a program 
can have an impact across all of the domains. It mandates the development of 
HSI criteria for all programs. Enclosure 1 cites 20 instructions and industry 


standards as references to assist in developing and evaluating HSI trade-offs. 


4. Conclusions on United States Navy Human Systems 
Integration (HSI) Guidance 


The Navy’s HSI guidance expands the scope to include modernization 
programs of legacy systems. Of all the DoD and U.S. Navy instructions reviewed, 
NAVSEA 3900.8A is the only one that mentions trade-offs across all domains. 


D. ACADEMIC HUMAN SYSTEMS INTEGRATION (HSI) GUIDANCE 


1. Handbook of Human Systems Integration (HSI) 


The Handbook of Human Systems Integration (Booher, 2003) is a 
965-page tome containing the most extensive guidance available on HSI. It is the 
collected work of approximately 90 contributors, technical advisors, and 
reviewers from the government, industry, and academia. Contributors reported 
the status of HSI in the United Kingdom, Canada, the United States Army, the 
United States Navy, the United States Air Force, the Federal Aviation 
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Administration, and the National Academy of Sciences. Many of the chapters 


cover both military and nonmilitary applications (Booher, 2003). 


Section 1.2.1 defines HSI as primarily a technical and managerial concept, 
with specific emphasis on methods and technologies that can be utilized to apply 
the HSI concept to systems integration (Booher, 2003). This circular definition is 
followed by a section that quotes the U.S. Army’s Manpower and Personnel 
Integration (MANPRINT) and DoDI 5000.2 definitions of the “HSI concept.” The 
preface describes HSI as a very attractive new integrating discipline that can help 
move business and engineering cultures toward a people-technology orientation 
(Booher, 2003). 


In the HSI Handbook, Booher describes HSI as an addition to the systems 


engineering (SE) process. The book cautions, 


If organizations are to change significantly to take full advantage of 
the benefits that HSI can offer, | [Booher] believe this is most likely 
to be accomplished as an inherent part of systems engineering and 
management. (Booher, 2003, p. xvi) 


Though HSI developed from previous HFE and SE work, it is more than 

just HFE or “good” systems engineering. Booher states in the foreword: 
Human factors and ergonomics, while important components 

are not themselves sufficient because they do not cover other 

important human domains . . . . The major component currently 

missing from systems engineering and management is a detailed 
description of the principles and methods of human systems. The 

intent of the Handbook is to provide that component. (Booher, 

2003, p. xvi) 

Acknowledging the broad scope of HSI and the diversity of contributors, 
the handbook is designed to help users identify the sections of most interest to 
them. The chapters are grouped into sections and each section and chapter start 
with an overview (Booher, 2003). The chapters that are most relevant to this 
thesis are Chapters 8 and 17, which address interactions and trade-offs between 


all of the domains. 
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Chapter 8, written by Barnes and Beevis, starts by citing the DoD direction 
for HSI in military systems. It discusses human performance measurement 
issues related to system design, with a focus on those issues that require 


trade-offs. When discussing the development process Barnes and Beevis state, 


Since a single model or simulation that is in any sense complete is 
usually not available early in the design process, the HSI 
practitioner must rely on a combination of modeling, human 
experimentation, and simulation for each stage of development. 
(Booher, 2003, p. 237) 


In Section 8.7, Interactions Among HSI Domains, Barnes and Beevis add 
that when planning human performance measures for a system under 
development, it is important to remember that the various HSI domains are 
interrelated and that changes in design to improve one domain nearly always 
affect other domains (Booher, 2003). Supporting the present study they note that, 
“Although the need is obviously great, there is no well-established body of 
knowledge on HSI domain trade-offs” (Booher, 2003, p. 254). 


Chapter 17, by Rouse and Boff, addresses types of HSI investments that 
have intangible characteristics in addition to more tangible attributes. For 
example, investing in more training or higher personnel selection standards can 
result in less health and safety costs and higher morale and productivity. Rouse 
and Boff also note the difficulty in balancing trade-offs between short-term and 


long-term benefits (Booher, 2003). 


Chapter 17 recommends using a multiattribute utility theory (MAUT) model 
to allow each stakeholder to assign “costs” and “benefit” values to each system 
attribute and time. This allows for quantitative weights of qualitative things, 
making assumptions explicit and open to testing. Rouse and Boff found that use 
of the MAUT methodology documents underlying assumptions, making them 
explicit and open to review (Booher, 2003). Quantifying and comparing utility 


inputs can be difficult with large numbers of variables. Evaluating costs and 
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benefits is further complicated by investment costs spread over time, uncertain 
returns, multiple stakeholders with different values, and _ indirect and 


intangible returns. 


2. Human-System Integration (HSI) in the System Development 
Process: A New Look 


The United States Army and United States Air Force funded this 384-page 
report by the National Academy of Science’s Committee on Human-System 
Design Support for Changing Technology. The introduction defines HSI as a 
discipline concerned with ensuring that the characteristics of people are 
considered throughout the system development process, with regard to their 
selection and training, their participation in system operation, and their health and 
safety (Pew & Mavor, 2007). 


The authors propose a “risk-driven” approach that achieves cost 
effectiveness by focusing resources on high-risk aspects of a program, while 
deemphasizing resources spent on low-risk aspects (Pew & Mavor, 2007). The 
focus of this book is similar to the focus of Chapter 17 of the Handbook of Human 
Systems Integration. This book differs from Chapter 17 of the Handbook of HSI 
with respect to benefit considerations. Pew and Mavor contend that HSI risk 
trade-offs must be presented in terms of acquisition and life-cycle costs and 
benefits, as opposed to including intangible benefits (Pew & Mavor, 2007). They 
agree with Chapter 17 of the Handbook of HSI on the need for a “shared 
representation” that can make assumptions explicit and open to review (Pew & 
Mavor, 2007). 


In addition to supporting the present work in trade space development, 
this book reinforced the past choices and influenced the current choices of 
variables. In the conclusions to one of their examples, they noted that HFE 


concerns are often linked to MPT concerns (Pew & Mavor, 2007). This supports 
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the first design of the NPS trade space by Simpson (2006), with MPT and HFE 
as inputs that determine the levels of performance, risk, and the other 
HSI domains. 


The book defines two different types of risk: operational risk and program 
risk. Figure 1 outlines the types of risk. 


Operational Risk Program Risk 
Usability Risk Exceeding Cost 


Figure 1. Taxonomy of Risk 


Exceeding Schedule 


The book defines operational risk as safety and usability risk. Safety is the 
risk that users will cause damage or be injured while using the item. Usability risk 
is the risk that operators will not use the item fully or at all. Program management 
risk is the risk of exceeding budgeted cost and the risk of exceeding allotted 
schedule (Pew & Mavor, 2007). 


E. HUMAN SYSTEMS INTEGRATION (HSI) CONCLUSIONS 


One obstacle to effective HSI in DoD programs is the long-term and 
uncertain nature of life-cycle costs. While the literature suggests that HSI should 
consider operational risks and life-cycle costs, current DoD and Navy instructions 
only apply to the acquisition process, which evaluates performance based 
predominately on short-term program risk. The other main obstacle is that, 
despite discussions of the interconnections between the domains, the HSI 
literature focuses primarily on the individual domains of HSI. DoD instructions 
discuss HFE and ESOH separately from the other domains, while MPT tend to 
be grouped together. 
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Several factors propagate this condition. All of the HSI domains, with the 
exception of system survivability, are stand-alone fields that predate HSI. Most 
HSI practitioners have education and background in only one or two of the HSI 
domains. In DoD and private industry, HSI domains are often separate functional 
areas in different parts of the organizations. All of these factors highlight the need 
for “shared representations” that can take inputs from experts in one domain, and 
display and translate them into effects on the other domains and on total 


system performance. 
F. SAFETY LITERATURE 


Safety considerations have traditionally been one of the largest drivers in 
HSI trade-offs. As seen in the DoD and U.S. Navy instructions reviewed 
previously, safety is often discussed separately and often in greater detail than 
the other HSI domains. This is not surprising, as it was the cost of airplane 
mishaps during World War II that formalized HFE, a primary domain of HSI, into 
a discipline. Therefore, an evaluation of safety literature from the United States 


Navy and civilian industries is relevant to this thesis. 


A 2004 study by the Marine Accident Investigation Branch (MAIB) of the 
United Kingdom looked at 66 mishaps of merchant vessels operating near the 
coast of the British Isles. The study determined that having only two watch 
standers taking turns led to fatigue and inadequate performance. It 
recommended adding a third person to the rotation and having a lookout in all but 
clear weather with light traffic (MAIB, 2004). The larger watch team size of Navy 
ships does not prevent similar mishaps. In an opinion piece from Fathom 
magazine, Rear Admiral Finney discussed two almost identical collisions, each 
between a Navy destroyer (DD) and a merchant ship, in the same shipping 
channel, 30 months apart. He notes that, in both cases, the crew failed to follow 
established procedures, failed to communicate, and lookouts were missing 


required equipment (Finney, 1992). 
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Thus, while manning levels and personnel are factors in mishap rates, as 
the size of the crew increases, manning and personnel factors have less 
influence than factors such as team performance and workload. An investigation 
of manning onboard 18 U.S. Navy ships by the Naval Health Research Center 
found that manpower utilization had a stronger relationship with performance and 
illness levels than did the manning level (Dean, Harvey, Pugh, & 
Gunderson, 1979). 


As another example, research efforts by National Aeronautical and Space 
Agency (NASA) and the Federal Aviation Administration (FAA) in the late 1970s 
found that human error and teamwork failures, and not mechanical malfunction, 
were the major causes of industrial accidents. This led to the development of 
Crew Resource Management (CRM) training programs by commercial airlines. 
CRM programs teach nontechnical skills, such as teamwork and communication, 
to improve operational performance. Training programs include both initial 
training and regular refresher training. In addition to its current use by all major 
international airlines, the CRM concept is applied in military aviation, 
anesthesiology, air traffic control, the Merchant Navy, the nuclear power industry, 
aviation maintenance, and the offshore oil industry (Flin, O’Connor, & 
Mearns, 2002). 


G. NPS HUMAN SYSTEMS INTEGRATION (HSI) CONCEPT 


This thesis builds on a theoretical framework proposed in an NPS thesis, 
“Toward an Improved Method of HSI Evaluation in Defense Acquisition” 
(Simpson, 2006). The previous thesis proposed a model for an HSI trade space 
tool (TST) with four input variables and four output measures. Manpower, 
personnel, training, and HFE are the four input variables; the remaining HSI 
domains are functions of the interactions of the inputs and the output is total 


system performance. 


The model represented in Figure 2 looks at how total system performance 
is determined by inputs of MPT and HFE with enabling/constraining factors of 
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cost, schedule, and risk. The final outcome, total system performance, is the 
combination of all of the first order outcomes, which includes mishap rates 
(Miller & Shattuck, 2007). 


Total 
System 


Performance 


© Miller and Shattuck, 2007 


Figure 2. NPS HSI Process Model (From Miller & Shattuck, 2007) 


From this model, a tradespace tool (TST) was developed by the NPS 
Office of Continuous Learning (OCL). The initial version of the TST was 
developed during thesis efforts by Simpson (2006). Figures 3 and 4 show the 
input screens for the NPS HSI TST. 
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Figure 3. NPS HSI TST Input Screen 


The user defined the relationships between four variables with each other, 
and with the acquisition variables of cost, schedule, performance, and risk. Once 
these 22 interactions were defined, the user progressed to a second screen, 
shown in Figure 4. This screen contained eight slider bars, four for the input 
variables and four for the output variables. Moving any one of the input slider 


bars shifted all four of the output bars according to the defined relationships. 


20 


Adjustable Pararreeters Enablers/Constraints Rest 


on ee ‘ 7s tar ter 982 


: Fy z : j : 3 : 
z ; ; 2 
PoF § 3 : 
z 
Inst 2 rE 
Figure 4. NPS HSI TST Results Screen 


H. NAVY REPORT FORMATS 


This section describes the format for the two data sets collected, U.S. 
Naval Safety Center Mishap Reports and SORTS reports. 


i. Naval Safety Center Mishap Reports 


The Chief of Naval Operations Instruction (OPNAVINST) 5102.1D (2005) 


defines Class A, B, and C mishaps in this way: 


1. Class A Mishap. The resulting total cost of damages to DoD or 
non-DoD property in an amount of $1 million or more; a DoD 
aircraft is destroyed; or an injury and/or occupational illness result 
in a fatality or permanent total disability. 


2. Class B Mishap. The resulting total cost of damages to DoD or 
non-DoD property is $200,000 or more, but less than $1 million. An 
injury and/or occupational illness result in permanent partial 
disability or when three or more personnel are hospitalized for 
inpatient care (beyond observation) as a result of a single mishap. 
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3. Class C Mishap. The resulting total cost of damages to DoD or 
non-DoD property is $20,000 or more, but less than $200,000; a 
nonfatal injury that causes any loss of time from work beyond the 
day or shift on which it occurred; or a nonfatal occupational illness 
that causes loss of time from work or disability at any time 
(OPNAVINST 5102.1D, 2005, p. 2-1). 


OPNAVINST 5102.1D requires investigation and reporting of all Class A, 
B, and C mishaps. It does not define a Class D mishap. Based on the definitions 
listed above, a Class D mishap could be defined as an incident that resulted in 
less than $20,000 of damages or in a nonfatal injury that did not cause any loss 
of time from work beyond the day or shift on which it occurred. The OPNAVINST 
also directs that “near mishaps should be investigated and reported via a 
HAZREP [Navy message]” (OPNAVINST 5102.1D, 2005, p. 1-12). The mishap 
reporting matrix specifically directs a hazard report for cases of “Electrical shock 
afloat from a design deficiency (no injury)” and “Man-overboard mishap 
underway with no injury” (OPNAVINST 5102.1D, 2005, p. 5-8). The OPNAVINST 
provides discretion to unit commanding officers on whether or not to report 


Class D mishaps and near misses. 
Zs Status of Resources and Training System (SORTS) Reports 


Each U.S. Navy ship submits a monthly SORTS report. It contains five 
columns for reported status: overall status, personnel status, equipment condition 
status, supply/equipment status, and training status. Each reported status ranges 
from best, (Category One or C-1), to worst, (Category Five or C-5). The following 
excerpt from Commander, Military Sealift Command Instruction (COMMSCINST) 
3501.2B (1998) defines each of the levels for the overall category. 


5.4.1 Category Levels Defined 


The overall unit category level will be based only upon organic 
resources and training under the operational control of the reporting 
unit or its parent unit. The five categories of overall unit category 
levels are: 
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a. C-1. Unit possesses the required resources and is trained to 
undertake the full wartime mission for which it is organized or 
designed. 


b. C-2. Unit possesses the resources and has accomplished the 
training necessary to undertake the bulk of wartime mission for 
which it is organized or designed. 


c. C-3. Unit possesses the resources and has accomplished the 
training necessary to undertake major portions of the wartime 
mission for which it is organized or designed. 


d. C-4. Unit requires additional resources and/or training to 
undertake its wartime mission, but if the situation dictates, it may be 
directed to undertake portions of its wartime mission with resources 
on hand. 


e. C-5. Unit is undergoing a service-directed resource change and 
is not prepared, at this time, to undertake the wartime mission for 
which it is organized or designed. C-5 units are restricted to: 


(1) Ship in overhaul or restricted availability for 30 days or longer. 
(2) Units undergoing major equipment conversion/ transition. 

(3) Units placed in cadre status by the parent service. 

(4) Units being activated, inactivated and reactivated. 


(5) Units not manned or equipped but required in the wartime force 
structure. 


(6) Units tasked as training units that could be tasked to perform a 
wartime mission. 


5.4.2 Category Levels, Caveats 


The overall unit category level will be identical to the lowest level 
[highest number] recorded in any of the unit's individually measured 
resource areas of personnel, equipment and supplies on hand, 
equipment condition and training, unless subjectively raised or 
lowered by the unit commander. However, modification of a unit's 
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overall level by its commander does not permit modification of the 
computed status of each individually measured area, which must 
be reported without adjustment. (COMSCINST 3501.2B, 1988, 
pp. 15-16) 


All U.S. Navy ships undergo overhaul and maintenance from time to time. We 
wanted to focus the model for this thesis on only those months in which ships 
were in operational status. However, it was not possible to determine exactly 
when a ship was in a scheduled maintenance period. Based on the definitions of 
the overall category, some ship scheduling information can be derived from the 
SORTS report. Therefore, those months when a ship reported a C-5 rating were 
attributed to planned maintenance activities, rather than lack of manning, 


insufficient training, or equipment malfunctions. 
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lll. APPROACH 


A. OVERVIEW 


The initial version of the TST, developed by NPS’s Office of Continuous 
Learning (OCL) and based on the model proposed by Miller and Shattuck (2007), 
had 22 user-defined correlations. The four predictor variables were manpower, 
personnel, training, and HFE. The user defined the interactions of these four 
inputs with each other, and with the acquisition variables of cost, schedule, 
performance, and risk. Since it was not possible to obtain sufficient data on all 
variables of interest, this thesis used manning, SORTS, and mishap data to test 


whether there were relationships among the variables that were available. 


Section B identifies the variables tested in this thesis. Section C states the 
hypotheses tested. To test the model, data were collected on the U.S. Navy’s 
Oliver Hazard Perry Class Frigate (FFG-7). Section D provides background 
information on the FFG-7 class. Section E details the process used to collect the 
data. Section F describes how the obtained data were cleaned and prepared for 


analysis. Section G describes how the data were used to quantify each variable. 
B. VARIABLE DEFINITION 


The predictor variable analyzed in this thesis was manning level. 


The outcome variables analyzed in this thesis were performance (SORTS) 


and safety (mishaps). 


Variables for which insufficient data were obtained (e.g., personnel and 
training) and those not applicable to this system (e.g., acquisition schedule), 
were not investigated in this thesis. Limiting the data collection to the FFG-7 
class was an attempt to hold other variables, such as HFE, constant. Data 
sources and information gathered on some of the controlled variables 
(e.g., Manning cost) are discussed in Chapter VI, under improvements to HSI 


data. 
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C. HYPOTHESES 


The purpose of this thesis is to use historical data to explore the nature of 
the relationships that are embedded in an existing notional model of HSI. Only 
those relationships for which there were appropriate and sufficient data were 
analyzed. The hypothesized relationships are: 


e There is a negative relationship between manning and SORTS 
levels (i.e€., aS manning increases, performance as measured on 
SORTS will improve). 


e There is a negative relationship between manning and mishaps 


(i.e., aS manning levels go down, mishaps will increase). 


e There is a positive relationship between mishaps and SORTS 
levels (i.e., aS mishaps increase, performance as measured on 
SORTS will decrease). 


D. USN FRIGATE OVERVIEW 


The scope of this thesis depended on two major decisions. The first 
decision was which class of naval platforms to evaluate. The Navy fact files at 
http:/Awww.navy.mil/navydata/fact.asp provide basic information on aircraft and 
ship types. As discussed in Chapter I, this thesis focused on the Navy “long hull” 
Frigate FFG-7 class. The second decision was the time period for the data 
request. The United States Navy commissioned the last ship of the class, the 
USS Ingraham, (FFG-61), in August 1989; thus, the initial requests were for data 
from October 1989 to October 2006. This time period was later reduced to 
October 1994 through September 2006, due to lack of available data. The end 
result was 96 months of data for each of the 30 U.S. Navy Frigates active at the 


time. 


Bath Iron Works and Todd Shipyards built 55 FFG-7 class ships in two 
major variants. The United States Navy received 51 and the Australian Navy 


received four of these ships. The “short” class was 445 feet long. The “long” 
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class was 453 feet long (FAS, 2000). A list of commissioning and 
decommissioning dates for all Navy Frigates was created, based on information 
from two private Websites: the “Unofficial U.S. Navy Site” at 
http://navysite.de/frigates.htm and “Federation of American Scientists, Military 
Analysis Network” at http:/Avww.fas.org/man/dod-101/sys/ship/ffg-7.htm. The list 
of decommissioning dates was compared to the official Navy FFG fact sheet 
Website at http://www.navy.mil/navydata/fact_display.asp?cid=4200&tid=1300& 
ct=4. The Navy fact sheet does not include decommissioning dates, but the list of 
decommissioned ships corresponds to all of the “short” class FFGs. All of the 
“long” class FFGs are listed as currently active. The Naval Manpower Analysis 
Center (NAVMAC) and the FFG Class Squadron (CLASSRON) confirmed this 
information and provided a list of Unit Identification Codes (UICs) for each 
active FFG. 


E. DATA COMPILATION METHOD 


This section describes the process used to collect the data. Locations of 
contact information are described, rather than using specific names, phone 
numbers, and e-mail addresses. This was done for two reasons: military 
personnel change jobs frequently and it aids the research performed on various 


Navy platforms. 
1. Homeports 


The homeport of each active FFG is on the official Navy FFG fact sheet 
Website at http://www.navy.mil/navydata/fact_display.asp?cid=4200&tid=1300& 
ct=4www.navy.mil. As of November 2008, this Website was last updated in 
October 2007. The locations of the 30 ships are discussed in Chapter IV, 
Results. 


27 


2. Unit Identification Codes (UICs) 


A listing of FFG UICs was necessary to obtain the manning and SORTS 
data. The FFG Class Squadron (CLASSRON) did not have historical manning or 
SORTS data, but provided a list of UICs for all active duty FFGs. 


The initial contact with the FFG CLASSRON was by telephone. The 
administrative department of Commander, Naval Surface Forces, Pacific (CNSP) 
provided the phone number. Subsequent communications were conducted via e- 
mail. The CLASSRON for FFGs is Destroyer Squadron 14 out of Mayport, 
Florida. Contact phone numbers are located on the bottom of the “Command” 


page of their homepage at http:/Awww.cds14.surfor.navy.mil/default.aspx. 


To find contact information for other Navy ship classes, the Commander, 
Naval Surface Forces, Pacific Website contains a page of all Surface Navy 
CLASSRONSs at http://www.surfpac.navy.mil/site%20pages/classrons.aspx. This 
page contains lists of links to the homepage of each ship, within each 
CLASSRON. The CNSP Website does not have links to the CLASSRON’s 
homepages. The administrative department of CNSP may have phone numbers 
for different CLASSRONs. Commander Navy Surface Forces, Pacific (CNSP) 


lists their phone number on their homepage at 


http://www.surfpac.navy.mil/default.aspx. 
3. Authorized Manning Level 


The Naval Manpower Analysis Center (NAVMAC) did not have historic 
manning or SORTS data, but provided three “Ship Manpower Document” (SMD) 
reports. The most recent SMD was prepared in January 2004, and is currently 
under revision by NAVMAC. The other SMD reports were prepared in December 
1997 and September 1999. The reports detail every authorized billet by 
department, division, rank, rate, officer designator, Navy Enlisted Classification 
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(NEC) code, and officer subspecialty code. This allowed for the calculation of 
percent of manning compared to authorized billets, compensating for changes in 


the authorized levels over time. 


The initial contact with NAVMAC was by telephone. Subsequent 
communications were conducted via e-mail. The Navy Personnel Command 
(NPC) customer service line provided NAVMAC’s phone number. NPC’s toll-free 


customer service number is located on the NPC homepage at 
http:/Mwww.npc.navy.mil/Channels/. 


4. Manning Cost 


The Chief of Naval Operations Programming Division, N80, provided the 
Active Duty Programming Rates for FY 2008 by e-mail. Contact information is 
published in the “OPNAV Planning, Programming, Budgeting and Execution 
(PPBE) and Program Budget Information System (PBIS) Courses for Fiscal Year 
2009 memorandum,” which is posted on the N8 homepage under “N8 Hot Topics 
(Public)”  (https://ucstcdom03.ahf.nmci.navy.mil/n8/webbas01.nsf/(vwwebpage)/ 
home.htm?OpenDocument). Alternative cost measures are discussed in 


Chapter V, under recommendations for future research. 
5. Safety Mishap Data 


The initial contact with the Naval Safety Center was by telephone. The 
Naval Safety Center, Afloat, Surface Warfare Division Website, at 
http://www.safetycenter.navy.mil/afloat/surface/index.asp, lists the Division 
Head’s phone number. Subsequent communications were via phone and e-mail, 


with e-mail being the primary method. 


The Naval Safety Center provided an unclassified Microsoft Excel 2003 
document by e-mail. The Excel table gave the date and description of all mishaps 
onboard Navy FFGs, from FY 1998 through FY 2006. The columns of the data 
included Event Serial Number, Ship UIC, fiscal year, calendar year and month, 


and mishap class (A, B, C, or D). These data were used as a measure of safety. 
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6. Manning and Personnel Data 


The manning, personnel, and SORTS report data were obtained from the 
Defense Manpower Data Center (DMDC) via their online Data Request System 
(DRS) at https:/Mwww.dmdc.osd.mil/drs/. The initial request was for all Enlisted 
Data Verification Report (EDVR) and SORTS reports on a monthly basis from 
October 1989 to October 2006 for all Navy FFGs. Following the online 
submission, DMDC made contact via e-mail and requested phone contact to 
clarify the request. They required a list of UlCs to access the data requested. 
Due to the nature of the request, it was split in two datasets (one unclassified and 


one classified), and completed by two different DMDC workers. 


DMDC provided unclassified monthly manning and personnel data from 
October 1994 to June 2008, in a comma separated text file. The DMDC database 
did not have monthly manning data prior to October 1994. The file contained six 
columns: date (YYMM format); rank (E01-E09, W01-W05, 001-009 format), 
rating, (two to four letters for enlisted ratings, plus four-digit NEC code, if 
applicable, and four-digit code for officer designators), years of service, ship UIC, 
and count (number of people with that set of qualities). The file contained 
905,323 rows of data. In most cases, there was only one person on board the 
ship that month with the exact same combination of rank, rate/NEC, and years of 
service. However, some categories, such as undesignated E01s with less than 
one year of service, had as many as 12 individuals with the same characteristics, 
which is why there was an average of 184 rows of manning and personnel data 


per ship, per month. 
7. Performance Data 


DMDC provided the SORTS data, classified SECRET, from January 1989 
through September 2006 via secure e-mail. The Microsoft Excel 2003 document 
had 12 columns including year (four-digit format), month (two-digit format), 
component (active or reserve), and UIC. It also contained five columns of that 


month’s reported status: overall status, personnel status, equipment condition 
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status, supply/equipment status, and training status. As discussed in the 
literature review, each reported status is inverted in order of performance and 
ranges from Category One (C-1), best, to Category Five (C-5), worst 


performance. 
F. DATA CLEANING 


We obtained three and created two separate sets of data for each ship, for 
each month, which were merged to create one data table. All of the obtained 
data files contained missing data and duplicate entries. This section describes 
how each of the data tables was cleaned or created. The manning and personnel 
data set started as a comma Separated text file. The other data sets provided or 
created were Microsoft Excel 2003 files. Unless otherwise noted, all of the 
cleaning and analysis was done with the statistical software program JMP® 
version 7.0.2 (SAS Institute Inc., 2007). JMP can import a wide range of files, 


including comma separated text files and Microsoft Excel files. 
1. Homeport Data Table Creation 


The UIC list was amended by hand with Homeport and Region columns. 


This allowed for exploration of possible clustering of the data. 
2. Mishap Data Cleaning 


Some mishaps involved more than one “Mishap Type” category, 
e.g., “impact with object” and “flooding” may have occurred in the same mishap 
and would have created two row entries for one Mishap Serial Number. Since the 
“Mishap Type” and the event description columns were not of interest for this 
thesis, these two columns were deleted. Next, the duplicate rows were deleted. 
This prevented duplicate counting of mishaps when the data were summed by 


month and ship. 
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3. Manning Data Set Cleaning 


An exploratory pivot table of the Manning data for one ship in Microsoft 
Excel 2003 returned unreasonably large numbers: for total ship manning: over 
400 crewmembers for eight consecutive months. Clicking on one of the suspect 
cells in the pivot table generated a new spreadsheet with a copy of the source 
data. Sorting the data by Rate/NEC, then by years of service, revealed a pattern 
that suggested rows containing O00 or EOO in the rank field, which represented 
missing data, were duplicate entries. Nearly 46,000 out of over 905,000 lines of 
the Manning data appeared to be duplicates. These suspect rows were excluded 


from subsequent data analysis. 
4. Authorized Manning and Personnel Table Creation 


A table of the authorized manning and personnel levels was created by 
hand. The first column contained the list of the 96 months of the final data set, 
October 1998 to September 2006. The next four columns contained the 
“organizational manpower requirements” summary for that month from the 
applicable SMD obtained from NAVMAC. Table 1 shows a condensed version of 


the authorized manning table. 


Table 1. Historical Navy FFG Authorized Manning Levels 
Enliste 
Time Period Officer | CPO d Total 
Oct 1998 to Sep 1999 16 17 188 221 
Oct 1999 to Jan 2004 17 19 198 234 
Feb 2004 to Oct 2006 17 17 181 215 


In December 1997, the authorized crew level was 16 officers, 17 Chief 
Petty Officers (CPOs) and 188 “other enlisted,” for a total crew size of 221 
individuals. In September 1999, the authorized crew was 17 Officers, 19 CPOs, 


and 198 other enlisted, for a total crew of 234 individuals. In February 2004, the 
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authorized crew size was_ reduced to 17 officers, 17 CPOs, and 171 other 
enlisted, for a total crew of 215 individuals. Section G.2 of this chapter discusses 


the 2004 reduction in crew size. 


Ds Status of Readiness and Training System (SORTS) 
Data Cleaning 


An exploratory pivot table of the raw SORTS data in Microsoft Excel 2003 
identified three possible issues. The table defined the Row inputs as Year and 
Month, the Column inputs as UIC, and the intersections as the “overall” report. 
The first issue raised was that the data set contained numerous holes, or missing 
entries. The second issue was that the data set did not contain any entries for a 
few months; that is, there were no data for any of the ships. The third issue was 
that some entries in the table were between 6 and 10, yet the raw data did not 
have any values above 5. Sorting the data by year, then by month, then by UIC 
uncovered nearly four dozen duplicate entries. The duplicate entries were 


excluded from the data analysis. 
G. DEFINITIONS OF PREDICTOR AND OUTCOME VARIABLES 


This section describes how the variables were quantified, using the data 
obtained. It is neither inclusive nor exhaustive of data sources or assessment 
methods. Due to data limitations, the time period analyzed was trimmed to 
October 1998 through September 2006. 


Eight fiscal years (FYs), or 96 months of data on 30 ships, produced 2,880 
rows in the final table. Since all of the three data tables contained missing 
entries, a two-column table was created in Excel and imported into JMP. This 
2,880-row table contained every unique combination of dates and UICs. Then we 
merged the 2,880 table with each of the other data tables using JMP. By 


including nonmatches, JMP inserted rows of missing data as applicable. 
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a. Manning 


Manning is the total number of people assigned to the ship for a given 
month and was used as a Substitute for manpower. Manpower is the number of 
work hours that can be performed by the assigned crew. Due to changes in 
authorized manning levels, this thesis quantified manning as the percent of total 
authorized crew assigned to a ship for that month. The cleaned manning data set 
for October 1998 to October 2006 contained 497,489 rows. This data table was 
joined with the authorized manning data table. Summing by UIC and date 
produced one row per month, per ship. The summed data table had 30 missing 


values, resulting in 2,850 rows. 


2. Human Factors Engineering (HFE), Habitability, 
and Survivability 


It was beyond the scope of the current study to obtain HFE, habitability, or 
system survivability data. However, since all ships were from the “long” FFG-7 
class, they are assumed to have similar physical configurations. This study did 


not obtain detailed data on ship alterations and modifications. 


Open source information and the reduction of authorized manning by 
2 CPOs and 17 junior enlisted in the January 2004 Ships Manning Document 
imply that all 30 of the Navy’s FFG-7s underwent major ship alterations in 2003. 
The “Unofficial U.S. Navy site” states, “in 2003, the Navy started to remove the 
Mk-13 guided missile launchers from the remaining [FFG-7 Oliver Hazard] 
PERRY class frigates” (Doehring, n.d.). This concurs with the information on the 
“Destroyer History” Webpage that, 


2003 was a significant year for the [FFG-7 Class] Perrys in the US 
Navy. Raytheon terminated support for the obsolete SM-1MR 
missile. [The US Navy] embarked on a program not only to remove 
the missile launchers from the 30 long-hull ships but to modernize 
them, reducing their cost of operation and extending their service 
lives to up to 40 years. (Destroyer History Foundation, 2008) 
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Despite still referring to them as “guided missile frigates,” the “Official 
Navy FFG fact file” sheet does not mention the ships ever having had a missile 
launcher. “The guided missile frigates (FFG) bring a short range anti-air warfare 
(AAW) capability (provided by their Phalanx Close-In Weapon System) to the 
frigate mission” (U.S. Navy, 2007). 


= Safety 


Safety was quantified as the number and frequency of mishaps onboard 
U.S. Navy Frigates. As discussed in the literature review, reports are not required 
for most Class D mishaps. Based on this and preliminary data analysis, safety 


was quantified both with and without Class D events. 


Six safety variables were calculated: number of each class, total 
reportable mishaps (Class A, B, and C), and total number of mishaps (Class A, 
B, C, and D). The safety data table was transformed using the same method as 
the manning data table. This resulted in a data table with 329 rows. It was 
merged with the 2,880 table to align the entries to the other data tables, based on 
ship UIC and month. Since ships do not file monthly negative reports, it is not 


possible to know if any Class A, B, or C reports are missing from this data set. 
4. Performance 


This thesis quantified performance as the SORTS reported levels. Five 
SORTS variables were provided: overall status, personnel status, equipment 
condition status, supply/equipment status, and training status. As discussed 
previously, SORTS levels are low when performance is high, i.e., a ‘1’ on 
SORTS corresponds to the maximum rating of performance. Merging the SORTS 
data table with the 2,880 table filled in approximately 400 missing rows. 
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IV. RESULTS 


This chapter is divided into three sections. The first section is descriptive 
statistics, the second section is the correlation analysis, and the third section is 
an analysis of manning and mishap rankings across all ships. Three data tables 
(manning, safety, and SORTS) were merged into a single data table (described 
in Table 2). The columns of date and UIC were used to align the data. The data 
set contained 2880 rows, one for each ship for each month, with 17 columns per 


row. 


Table 2. Variables in Data Set 


Date (month and year) 
UIC (e.g., N21032) 
Component (active or reserve) 
Ship location 
Homeport (City and State) 
Region (e.g., East Coast) 
SORTS scores (1 to 5) 
Overall 
P, (personnel) 
R, (equipment) 
S, (Supplies) 
T, (training) 
Mishaps (counts) 
Class A 
Class B 
Class C 
Class D 
Class A-C 
Class A-D 
Percent total manning 


The date and manning data were encoded as continuous variables in 


JMP. All other variables were encoded as ordinal or categorical data. 
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A. DESCRIPTIVE STATISTICS 


1. Component 


Of the 30 ships, the SORTS data identified one ship as belonging to the 
U.S. Navy Reserve (USNR) and the rest of the ships as active component 
(USN). 


2. Homeports 


The Navy’s FFGs operate out of six homeports. The largest concentration 
is in Mayport, Florida, with 12 ships. Combined with the six FFGs stationed in 
Norfolk, Virginia, there are 18 FFGs stationed on the East Coast of the United 
States. San Diego, California, has five FFGs, and Everett, Washington, has three 
FFGs, for a total of eight FFGs on the West Coast. Pearl Harbor, Hawaii and 
Yokosuka, Japan both have two FFGs, for a total of four FFGs outside the 
continental United States (OCONUS). Table 3 lists the number of ships by 


homeport and region. 


Table 3. | Navy FFGs by Homeport and Region 


# of # of 

HOME PORT Ships REGION Ships 
M ayp art FL 12 East 18 
N orfok VA 6 Coast 
San Diego CA 5 West 0 
Cyereat VA J Coast 
PearlH arbor HI 2 OCONUS 4 
Yakosuka 2 
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3. SORTS Report 


The descriptive statistics of the SORTS reports, classified as SECRET, 
are not included in this thesis pending final determination of classification. No 
significant relationships were found between any of the SORTS scores and any 


of the other variables. 
4. Mishap Report Rates 


Table 4 shows the descriptive statistics for the mishap reports for all 30 
ships over 96 months. The monthly rates are average mishap rates per ship per 
month. The number of mishaps reported by any ship over the 96 months ranged 
from three to 32; the average number of mishaps was 14.17 while the most 
common number of reported mishaps was 12. Table 4 includes statistics for the 
subtotal of all reportable mishaps (Class A, B and C). As mentioned in Chapter Il, 


Literature Review, reporting of Class D mishaps is optional. 


Table 4. | Mishap Statistics Per Ship Over Eight FYs 


Minimum Maximum Average Std Dev Mode Total Monthly 
Rate 
Class A 0 1 0.20 0.41 0 6 0.0021 
Class B 0 2 0.53 0.73 0 16 0.0056 
Class C 1 13 5.50 3.10 8 165 0.0573 
Class D 1 24 7.93 6.13 8 238 0.0826 
Class A-C 1 14 6.23 3.30 5 187 0.0649 
Class A-D 3 32 14.17 6.70 12 425 0.1476 


oe Manning 


Figure 5 shows the percent manning over the 96 months for each of the 
30 ships. The manning level for any ship for any month ranged from 42.74% to 
121.40% with the mean manning level at 92.72% and median of 94.87%. 
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Percent Manning 


Figure 5. Percent Manning By UIC for All 30 Ships 


The three ships circled in red Figure 5 have significantly lower manning 
levels than the other 27 ships. One of the three ships was confirmed to be 
Reserve Component while the other two are suspected to be Reserve 
Component. These three ships were determined to be sufficiently different from 


the others and, therefore, were excluded from further analysis. 


Figure 6 shows the percent manning over the 96 months for each of the 
27 ships believed to be active component. The manning level for any ship for any 
month ranged from 64.96% to 121.40% with a mean manning level of 96.28% 


and a median of 95.93%. 


— 


Percent Manning 


Figure 6. Percent Manning by UIC for 27 Ships 
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B. CORRELATION MATRIX 


A matrix of Pearson correlations for percent manning and the six safety 
variables (Classes A, B, C, D, Class A through C and Class A through D) was 
calculated in Microsoft Excel for the 27 ships in the study sample. Table 5 shows 
this correlation matrix. Another version of this matrix, including additional 
exploratory variables, is in Appendix A. All of the correlations in Table 5 were 
Statistically significant at the 0.05 level. Not surprisingly, the correlations between 
several of the mishap classes is fairly high. Of interest to this thesis is the 
relationship between manning and mishaps. 


Table 5. Correlation Matrix 


& &% % & Se “Gq 
2s. 85, 9s, %, 88, 
4q } © } © 

PercentManning 0.0005 0.0062 0.0295 0.0253 0.0295 0.0372 
Class A 1.0000 -0.0035 0.0485 0.0102 0.2106 0.1327 
Class B 1.0000 0.0038 -0.0165 0.2557 0.1385 
Class C 1.0000 0.0488 0.9534 0.6025 
Class D 1.0000 0.0436 0.8064 


Class A-C 1.0000 0.6260 


C. ANALYSIS OF RELATIVE RANKING BY SHIP 


To explore general trends in manning and mishaps, all of the ships were 
ranked from 1 (best), to 27 (worst), for each month of the study period based on 
percent of total manning and number of reportable mishaps. The ship with the 
highest percent manning for a month was assigned a manning rank of 1 while the 
ship with the lowest percent manning for a given month was assigned a rank of 
27. The ship with the most reportable mishaps for a month was assigned a 
mishap rank of 27. If a ship did not have any mishaps for a month it was 


assigned a mishap rank of 0, (i.e. if only three ships reported mishaps for a 
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month, then the top 24 ships were all assigned the mishap rank of O and the 
other three were assigned mishap ranks of 25, 26 and 27). Table 6 shows the 
results of averaging each of the two ranks over the 96 months then ranking the 
ships again based on their averages. The column on the left is the average of the 
average manning and average mishap ranks over 96 months for each ship. The 
right three columns are the corresponding averages converting into a rank from 1 
to 27. The rows are sorted from lowest average overall rank to highest average 
overall rank. The three highest ranks are in bold and the three lowest ranks are 


underlined and in bold. 


Overall Average Average Overall Manning Mishap 
Average Manning Mishap Rank Rank Rank 
3.0573 5.5625 0.5521 1 1 3 
3.8073 5.6667 1.9479 2 2 22 
4.4844 7.8750 1.0938 3 3 10 
5.3125 9.2708 1.3542 4 4 13 
5.3542 10.4271 0.2813 5 5 2 
5.8906 10.6979 1.0833 6 6 9 
6.2604 10.8646 1.6563 7 7 17 
6.5990 11.2292 1.9688 8 8 23 
6.6354 12.1667 1.1042 9 9 12 
6.8490 12.3229 1.3750 10 10 14 
6.9479 12.8438 1.0521 11 11 6 
7.5781 14.0833 1.0729 12 12 8 
8.1563 14.6354 1.6771 13 13 19 
8.2396 16.2188 0.2604 14 18 1 
8.3594 16.1667 0.5521 15 17 4 
8.4219 16.0104 0.8333 16 15 5 
8.5104 15.9271 1.0938 17 14 11 
8.8594 16.0938 1.6250 18 16 16 
9.2135 17.3750 1.0521 19 22 7 
9.2917 17.1979 1.3854 20 20 15 
9.4688 17.0104 1.9271 21 19 20 
9.6667 17.3958 1.9375 22 23 21 
9.6979 17.2396 2.1563 23 21 24 
10.2813 =17.8125 2.7500 24 25 26 
10.3906 17.5313 3.2500 25 24 27 
10.4531 19.2500 1.6563 26 27 18 
10.9219 19.1250 2.7188 27 26 25 


Table 6. Relative Ranking of Ships by Manning and Mishaps 


As is evident in Table 6, ships with low overall ranks tend to have both 
relatively low manning levels and relatively high mishap rates. 
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Figure 7 compares manning rank over time with the mishap rank over time 
for the ships with the two highest and two lowest overall ranks. Figure 8 shows 
the same data for only two ships: the highest and lowest in ranking. Figures 7 
and 8 demonstrate that manning ranks for all of the ships varied over time. No 
ship was consistently ranked high or low over the entire 96 months. 
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Figure 7. Manning and Mishap Rankings Over 96 Months for the Top Two 
and Bottom Two Overall Ranked Ships. 
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Figure 8. Manning and Mishap Rankings Over 96 Months for The Top and 


Bottom Overall Ranked Ships. 


To determine whether relative ranking in manning level is independent of 
mishap rankings, a nonparametric correlation was performed in JMP. The 
manning rank and mishap rank columns in Table 6 were compared using 
Spearman’s rho. The variables were positively correlated (Spearman’s rho = 
0.4194, p-value = 0.0294). This finding shows that there is a positive relationship 
between a ship’s average manning rank and its average mishap rank over the 96 
months. In other words, there is an indication of a negative relationship between 


manning levels and mishap rates. Mishap rates increase as manning levels drop. 
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V. DISCUSSION 


The first section of this chapter discusses the three hypotheses based on 
the results reported in the previous chapter. The second section lists the 
conclusions drawn from the analysis of relative rankings. The third section 
discusses the implications of the results with respect to the NPS HSI TST 


computer program. 


A. HYPOTHESES 


aE Hypothesis One, Manning and Performance 


The results failed to support hypothesis one, that there is a positive 
relationship between manning and SORTS levels. There were no significant 


relationships between percent manning and any of the five SORTS reports. 
2. Hypothesis Two, Manning and Safety 


The results of the rankings in Table 6 support hypothesis two, that there is 
a negative relationship between manning and mishaps. There was a statistically 
significant negative correlation between manning levels and mishap rates. Ships 
that have relatively low percent manning levels also have lower mishap ranks, 


which corresponds to higher rates of mishaps. 
a: Hypothesis Three, Safety and Performance 


The results failed to support hypothesis three, that there is a negative 
relationship between SORTS levels and mishaps. There were no statistically 
significant relationships between any of the eight safety variables and any of the 


five SORTS report variables. 
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B. MANNING AND MISHAPS 


Reportable mishaps (those involving serious injury or damages over 
$20,000) appear to be rare. This is good news for the U.S. Navy but provides a 
small data set for analysis. The analysis of relative rankings by ship found a 
Statistically significant negative relationship between percent manning and 
mishap rates. The results of this thesis support the idea that there is a negative 
relationship between manning levels and mishap rates, but we suggest that more 
detailed research is needed to isolate this relationship from numerous other 
influences and any noise that may be present in the data set. This is discussed 


more in Chapter VI, Conclusion and Recommendations. 
C. IMPLICATIONS FOR NPS HSI TST MODEL 


This thesis investigated two of the 22 relationships in the NPS HSI TST 
program. We found a statistically significant relationship between manning levels 
and mishap rates. However, the analysis of ranking did not derive an equation to 
estimate mishap rates based on percent of manning. The average rate of 
reportable mishaps (Class A, B and C) calculated from the safety data provided 
is 0.06 mishaps per ship per month. If, for example, the mishap rate for USN 
FFGs is not significantly influenced by the HFE level, then the average rate can 
be used to define the relationship between HFE and Safety in the NPS HSI TST. 
However, defining the mishap rate as a constant would make it irrelevant in 


tradeoff analysis. 


Although this was not pursued, the personnel costs provided by N80, 
$59,139 per year per enlisted and $123,269 per year per officer, could be used to 


define the relationships of cost with manpower and personnel. 
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VI. CONCLUSION AND RECOMMENDATIONS 


The first section of this chapter discusses how finding no significant 
relationship between many of the variables can justify a reduced sample size for 
future studies. The second section suggests ways to improve the data collection 
and analysis methods for each of the HSI domains. The third section suggests 
two possible directions to expand data collection beyond USN FFGs. The fourth 


section restates the implications of the results of this thesis. 
A. CONSISTENCY 


No practically significant differences were found in manning, personnel, 
performance or safety based on date, ships, homeport, or region. For example, 
there is no evidence that ships in Hawaii are more likely to report a class D 
mishap than ships in Norfolk, Virginia. This means that data collected on a 
sample of ships should be applicable to all Navy Frigates. However, there were 
significant differences in manning levels between one ship that was classified as 
a Navy Reserve ship in the SORTS data and 29 of the ships classified as Navy 
Active component. Two additional ships in the sample appeared to be Navy 
Reserve ships. The low manning level for these three ships may be due to 


exclusion of activated U.S. Navy Reserve personnel in the database. 
B. IMPROVEMENTS TO HSI DATA 


This section discusses possible improvements to the types of data 
collected, and analysis methods utilized, to explore HSI in the U.S. Navy. Future 
research may benefit from more detailed data collection. This thesis chose 
breadth (eight fiscal years) over depth of data (e.g., average hours spent on 
watch over one month of underway operations). The data analyzed for this thesis 
were insufficient to validate all the relationships in the notional trade space tool. 
However, empirical evidence was found to support a negative relationship 


between manning and mishap rates. This thesis did not provide substitutes for 
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missing data. The missing SORTS data could be estimated based on trends for 
the ship, but this may be less accurate, especially if several reports in a row are 


missing. 
1. Manpower 


The historical reports collected by the U.S. Navy do not record manpower 
data. Therefore, manning was used as a surrogate for manpower. The data 
collected could not distinguish between a crew that works 40 hours a week 
(which is a common practice while in port) from a crew that works over 80 hours 
a week (common on deployments) to maintain an overall SORTS score of 
Category 1. A better approximation would be to compare actual workload, 
collected onboard a sampling of ships, with the prescribed workload data 
contained in the SMDs provided by NAVMAC. A prospective study could be 
designed to do this type of analysis. Collecting these data retrospectively is 
virtually impossible. However, past manpower levels could be estimated by 
collecting and analyzing the following: ship’s schedules, typical watch stations 
manned based on ship’s schedule, personnel assigned, and maintenance jobs 
completed. Other aspects of manpower, such as unit cohesion and morale, could 


be estimated based on crew turnover rates. 
2. Personnel and Training Variables 


Eight personnel variables were analyzed but were deemed inadequate to 
test relationships between personnel and the other HSI domains. The variables 
were percent of junior enlisted (E01-E06), percent of Chief Petty Officers (CPOs), 
(E07-E09), percent of officers, percent of personnel with a Navy Enlisted 
Classification (NEC) code, and four summations of years of service. Chief 
Warrant Officers (W01-W05) were included in the officer category. Officer 


designators were evaluated as personnel without an NEC. 


This thesis proposed using NECs, rank, and years of service as personnel 


and training measures. The SMDs provided by NAVMAC and the data provided 
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by DMDC both included the same rank, rate, and NEC information. One could 
compare, by billet, the assigned personnel with the authorized personnel list. 
However, this method was beyond the scope of this thesis. A billet comparison is 
just one way to measure the personnel and training levels. Additional information 
sources could include an individual’s aptitude test scores at enlistment, grade or 


class standing at service schools, and advancement exam scores. 
3. Training 


Based on the data collected, it was not possible to quantify training. 
Training could be quantified as the activities that increase a unit’s level of 
personnel. In this way, a ship that sends a crewmember to school to obtain an 
NEC, achieves the same result as a ship that detaches an unqualified person 
and obtains a new person with the desired NEC. In both cases, the manning 


level stayed constant, while one measure of personnel increased. 
4. Schedule 


Schedule, as defined in the program acquisition process, normally only 
applies to delivery of new systems. Generally, it is not applicable to the 
evaluation of legacy systems, unless the system is completing a modernization 
program. Scheduling data, such as operational tempo and unit availability for 


tasking, is performance related and could be used in similar efforts in the future. 
5. Human Factors Engineering 


Due to the many similarities within the ‘long’ FFG class, HFE is difficult to 
assess within this community. HFE may be more useful for an analysis that 
includes several different classes (e.g., cruisers, destroyers and frigates). 
Several methods could test if HFE is a significant factor, to include determining 
the exact time period around 2003 when each ship had its missile launcher 
removed and other major upgrades performed. One could also obtain detailed 


information on every ship alteration and modification, such as the addition of a 
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new optic system, or upgrades to a computer system. It would be possible to 
obtain data for the decommissioned ‘short’ FFGs and investigate possible effects 


between these two distinct categories of ship platforms. 
6. Habitability, and Survivability 


As mentioned in Chapter Ill, Section G, this study did not obtain any 
detailed human factors engineering, habitability or system survivability data. The 
NPS HSI TST model theorizes that decisions made regarding manpower, 
personnel, training and HFE determine safety, habitability and system 
survivability. Just as training changes the level of personnel, changes in HFE 
impact the level of habitability and survivability. Therefore, any efforts to improve 
the measure of HFE should improve the measures of habitability and survivability 


as well. 
7. Risk/Safety Variable 


The results of this thesis identify safety as the variable that would most 
benefit from improved data collection and improved analysis. Due in part to the 
infrequent rate of all mishaps, this thesis did not analyze causal factors for any of 
the reported mishaps (e.g., mishaps due to heavy seas while underway were not 
evaluated separately from mishaps during command sponsored sports activities 
ashore). Board categories, included in the safety data from the Naval Safety 
Center, could be used to isolate only those mishaps suspected of being caused 
by manning or training factors. A better approach would be a detailed qualitative 
analysis of the full written mishap report for each mishap. Including minor injury 
and illness rates to future studies would increase the volume and sensitivity of 


safety data collected. 
8. Cost Variable 


Cost of ownership was not used in the data analysis, but some 


consideration was given to how to address the cost of a system. It is possible to 
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quantify manning and personnel costs as the FY 2008 Active Duty Programming 
Rates of $123,269 per year for officers and $59,139 per year enlisted. This thesis 
did not collect any cost estimates for training, HFE, acquisition, operations or 


maintenance. 


Developing an accurate method to determine total system cost is a goal 
worthy of its own study. One challenge in capturing the true cost of any DoD 
system is in defining the boundaries. In addition to payroll, a ship’s costs could 
include fuel, food, repair parts, shore based maintenance facilities, and medical 
care. For more information on alternate cost estimation methods see Naval 
Center for Cost Analysis (NCCA) ‘NCCA Cost Tools’ webpage at 


http://www.ncca.navy.mil/services/costtools.cfm. 


This thesis recommends quantifying manning and personnel cost as the 
Active Duty Programming rate for a single fiscal year. An alternative would be to 
obtain the manpower programming rates for each FY from the Program Budget 
Information System (PBIS) database or from N80, Programming Division. 


Separate rates for activated reservists are also available. 
9. Performance Variables 


All of the performance data used in this thesis were based on the SORTS 
data reported by each ship. The results of this thesis suggest that the SORTS 
scores are insensitive to the variables quantified. The SORTS scores are also 
somewhat subjective and confounded by the independent variables (e.g., if a 
ship does not have a critical NEC billet filled, it has to report a lower personnel 
level). Other performance measures that could be included in future studies are 
externally graded ship exercises, personnel reenlistment rates, time to earn 


watch station qualifications, operational tempo, and unit availability for tasking. 
Cc. ADDITIONAL APPLICATIONS 


The work of this thesis could be expanded to include additional ships that 


are similar to the USN Frigates, (e.g. Cruisers and Destroyers) to permit greater 
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pooling of data or it could be expanded to a variety of diverse naval platforms 


(e.g., amphibious ships, Submarines, or Aircraft Carriers). 
D. CONCLUSION 


HSI, though required by law, is often confused as just HFE or just SE. 
There remains a need for shared representations of the HSI trade space to 
facilitate Communication between what are often isolated domains. While the 
data collected for this thesis were insufficient to validate all the relationships in 
one notional trade space tool, finding empirical evidence of a negative 
relationship between manning levels and mishap rates is a substantial step 
forward. Just as this thesis advanced previous work, further research is needed 


to develop the NPS HSI Trade Space Tool to maturity. 
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CORRELATION MATRIX 


APPENDIX A. 
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This appendix contains an exploratory correlation matrix of 16 continuous 


variables from the 25 original variables. 
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APPENDIX B. DODI 5000.2 ENCLOSURE 7 


E7. ENCLOSURE 7, HUMAN SYSTEMS INTEGRATION (HSI) 


E7.1. General. The PM shall have a comprehensive plan for HSI in place early in 
the acquisition process to optimize total system performance, minimize total 
ownership costs, and ensure that the system is built to accommodate the 
characteristics of the user population that will operate, maintain, and support the 
system. HSI planning shall be summarized in the acquisition strategy and 
address the following: 


E7.1.1. Human Factors Engineering. The PM shall take steps (e.g., contract 
deliverables and Government/contractor IPT teams) to ensure human factors 
engineering/cognitive engineering is employed during systems engineering over 
the life of the program to provide for effective human-machine interfaces and to 
meet HSI requirements. Where practicable and cost effective, system designs 
shall minimize or eliminate system characteristics that require excessive 
cognitive, physical, or sensory skills; entail extensive training or workload- 
intensive tasks; result in mission-critical errors; or produce safety or health 
hazards. 


E7.1.2. Personnel. The PM shall work with the personnel community to define 
the human performance characteristics of the user population based on the 
system description, projected characteristics of target occupational specialties, 
and recruitment and retention trends. To the extent possible, systems shall not 
require special cognitive, physical, or sensory skills beyond that found in the 
specified user population. For those programs that require skill requirements that 
exceed the knowledge, skills, and abilities of current military occupational 
specialties or that require additional skill indicators or hard-to-fill military 
occupational specialties, the PM shall consult with personnel communities to 
identify readiness, personnel tempo (PERSTEMPO), and funding issues that 
impact program execution. 


E7.1.3. Habitability. The PM shall work with habitability representatives to 
establish requirements for the physical environment (e.g., adequate space and 
temperature control) and, if appropriate, requirements for personnel services 
(e.g., medical and mess) and living conditions (e.g., berthing and personal 
hygiene) for conditions that have a direct impact on meeting or sustaining system 
performance or that have such an adverse impact on quality of life and morale 
that recruitment or retention is degraded. 


E7.1.4. Manpower. In advance of contracting for operational support services, 
the PM shall work with the manpower community to determine the most efficient 
and cost-effective mix of DoD manpower and contract support. Once the 
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Manpower Estimate is approved by the DoD Component manpower authority, it 
shall serve as the authoritative source for reporting manpower in other program 
documentation. 


E7.1.5. Training. The PM shall work with the training community to develop 
options for individual, collective, and joint training for operators, maintainers and 
support personnel and, where appropriate, base training decisions on training 
effectiveness evaluations. The PM shall address major elements of the training 
system described in DoD Directive 1430.13, reference (bd), and place special 
emphasis on options that enhance user capabilities, maintain skill proficiencies, 
and reduce individual and collective training costs. The PM shall develop training 
system plans to maximize the use of new learning techniques, simulation 
technology, embedded training, and instrumentation systems that provide 
anytime, anyplace training and reduce the demand on the training establishment. 
Where possible, the PM shall maximize the use of simulation-supported 
embedded training, and the training systems shall fully support and mirror the 
interoperability of the operational system. For training programs that require 
training infrastructure modifications, the PM shall identify technology, schedule, 
and funding issues that impact program execution. 


E7.1.6. Environment, Safety and Occupational Health (ESOH). As part of risk 
reduction, the PM shall prevent ESOH hazards where possible, and _ shall 
manage ESOH hazards where they cannot be avoided. The acquisition strategy 
shall incorporate a summary of the Programmatic ESOH Evaluation (PESHE), 
including ESOH risks, a strategy for integrating ESOH considerations into the 
systems engineering process, identification of ESOH responsibilities, a method 
for tracking progress, and a compliance schedule for NEPA (42 U.S.C. 4321- 
4370d and Executive Order 12114, references (x) and (az)). During system 
design, the PM shall document hazardous materials used in the system and plan 
for the system’s demilitarization and disposal. The CAE (or for joint programs, the 
CAE of the Lead Executive Component) or designee, is the approval authority for 
system-related NEPA and E.O. 12114 documentation. For acceptance of ESOH 
mishap risks identified by the program, the CAE is the acceptance authority for 
high risks, PEO-level for serious risks, and the PM for medium and low risks as 
defined in the industry standard for system safety. 


E7.1.7. Survivability. For systems with missions that might require exposure to 
combat threats, the PM shall address personnel survivability issues including 
protection against fratricide, detection, and instantaneous, cumulative, and 
residual nuclear, biological, and chemical effects; the integrity of the crew 
compartment; and provisions for rapid egress when the system is severely 
damaged or destroyed. The PM shall address special equipment or gear needed 
to sustain crew operations in the operational environment. 

Last Modified on: 12/17/2004 (DODI 5000.2, 2004, p. E7). 
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